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A  TWO-DIMENSIONAL  SMOOTHING  SPLINE  AND  A  REGRESSION 
PROBLEM 

M.  ROSENBLATT 


INTRODUCTION 

Splines  have  been  proposed  in  a  variety  of  approxi- 
ration  problems  in  recent  years  (see  SCHOENBERG  14])- 
Smoothing  splines  as  a  particular  subclass  of  these  have 
drawn  particular  attention  (see  WAHBA  15])  in  the  context 
of  a  set  of  regression  problems.  It  had  also  been  noted 
that  boundary  effects  like  Gibbs  effects  can  arise  in  the 
case  of  natural  splines  (see  ROSENBLATT  I  3] ) .  A  few  re¬ 
marks  will  be  made  about  a  simple  one-dimensional  regres¬ 
sion  problem.  Suppose  that  observations 


*  =  /  (t  )  +  c . 

J  j  J 


j /n  ,  j  =  0,1,. . . ,n-l 


are  made  and  that  f  is  an  unknown  smooth  function  with 
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the  c .  random  errors 

i 


E  .  r  0 

Cl 


s  V  • 


A  cubic  smoothing  spline  estimate  g(t;X)  of  f(t)  is 
a  function  that  minimizes 


(1) 


~  T.  \x.-g(t.))7  +  X  /  (g"(t))2dt  . 

i  =  1 


The  asymptotic  behavior  of  such  an  estimate  g  of  f  in 
terms  of  the  mean  square  error 


(2) 


E  \  g  (t ;  \  )  - f  (t) 


has  been  examined  as  /)-*•«  .  One  determines  the  rate  at 
which  the  parameter  X  =  X  (n)  *0  as  n-»»  so  as  to  make 
(2)  go  to  zero  as  fast  as  possible  under  the  smoothness 
assumptions  made  on  f  .  The  c  .se  of  a  periodic  function 
/  and  a  periodic  smoothing  spline  has  been  discussed  in 
WAHBA  l  5]  and  RICE  and  ROSENBLATT  [  1J  .  In  the  latter 
paper  it  has  been  shown  that  boundary  effects  can  even 
arise  in  the  case  of  periodic  smoothing  splines.  The  non¬ 
periodic  case  of  a  smoothing  spline  is  treated  in  RICE 
and  ROSENBLATT  (  2]  .  The  boundary  effects  of  a  nonperiodic 


cubic  spline  are  examined  in  some  detail  in  this  last 


paper. 

There  are  many  measures  analogous  to  (1)  that  one 
could  consider  in  the  two  dimensional  case.  We  shall  not 
consider  one  that  leads  to  a  doubly  cubic  spline  but 
rather  a  simple'-  one.  Let  us  consider  observations 


xj,k  =  fUj'sk) 


+  E 


i 


t.  =  j/n,sk  =  k/n  , 


j  t  k  =  0 , 1 , . . .  , n-1 


with  f  an  unknown  smooth  function  and  the 
dom  errors  with 


’  3 


ran- 
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Ec 
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The  measure  that  will  be  minimized  to  obtain  the  smooth¬ 
ing  spline  approximation  g(t,s;X)  to  f  is 


n-1 


(3) 


\  E  l  9  (t  .,s  )  -x  k)2  + 
n  j ,k=C  J  J  ' 


1  1 


- ~rr  ■  ({Do}2  +  {D  f}2  )d  tds 


X 


with  D±  and  Dg  the  partial  derivatives  with  respect 
to  t  and  s  respectively. 

THE  SMOOTHING  SPLINE.  For  the  sake  of  simplicity  we 
shall  basically  consider  the  periodic  situation.  Let  us 
first  note  that  if  f  is  continuously  differentiable  up 

=  C2 )  for  k? 0 

-2u jt  -  2  tiki]  f  {t ,  i)  dtdi  = 

Ujt{f  (t  ,1) -f  (t  ,0)  )dt  - 

J  e  *T‘'ijt{x>T/(t,l)-£>i/(t/0)}dt  + 

\  \  -2  tilt  -  2«i*r  2  ,  .  .  .  _  , 

/  /  e  J  1>  f(t,‘ \)  d  tdi  . 

0  0 

The  corresponding  relation  for  is 

f  .  «  -  ;  e~2,iik'  {f  (l,T)-f  (0,T)  Id!  - 

J  ,  k  2kij 

.  1 

- ±___  ;  e~2,U*T  {d  /(1,t)-d  f  (0,t)  }  + 

[2r.ijr  0 

.  11  „  .  .  ... 

1  ,  ,  -2«nt-»T :ikt  .,2  ,  ,  .  ,  , .  , 

+  - j  J  J  e  J  D  f  (t  ,i)dtdi  . 

(2 rij)  0  0 


to  second  order  ( f  < 
1  1 

/.  =  J  /  exP{ 

J  oo 

■=Ai{ 

0 

1 

(2  rJ  k  )  ? 

♦  — r 

( 2  nik) 


If  both  7  and  k  are  not  zero  one  can  write 


j  ,k  2  tij 


1  r  !  e  2 { f  (1 ,  T ) -f  (0 , i )  )di  - 


1  -2ti7t-2*ilci__  ,  ,  ,  .  . 

(2777772771)  j.  J0e  DcV(t' T)dtdx 


Of  course,  the  corresponding  results  hold  for  g  if 


g  e  c*  . 

Consider  a  periodic  function  f  so  that 

(4)  f  (l.i)  =  t  (0,t)  , 

f  (t  ,1)  =  f  (t ,0)  . 


The  approximating  smoothing  spline  g  is  not  an  unre¬ 
stricted  function  minimizing  (3) .  We  assume  that  it  also 
satisfies  the  corresponding  boundary  conditions 

(S)  9(1,1)  =  g  ( 0  ,  i ) 

9 (t , 1)  =  g  (t ,0) 


and  minimizes  (3)  among  all  such  periodic  functions.  Let 
n-  i 

I]  x  exp(-2r, i  (iu  +  ftv)  )  , 

_  „  U  ,  v 
u  ,  v-  C 

and 
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j,k  u=v  J+un'*+vn 

Also,  given  the  conditions  (5) ,  it  follows  that  the 
Fourier  coefficients 


(6) 


jk 


(Oxg).k  =  (2%ik) g  jk  . 


The  expression  (3)  can  then  be  rewritten  as 

2 

I  + 


(7) 


n- 1 

j , k=0  J 


x  .  2 
-2-dLl 


+  X  £  £  {  j+un  I  +  Uc+vn  I  } \g  . 

.  ,  .  |37+un,Jt+vn 

j  ,  k=Q  u,v=-»  J 


The  expression  (7)  can  be  minimized  by  separately  minimiz 
ing  the  terms  for  each  pair  [j  ,k)  .A  minimum  is  ob¬ 
tained  for  (j  ,k)  =  (0,0)  by  setting 


g  =  0  if  (u,v)  4  (0,0)  . 

un ,  vn 


This  implies  that 


0,0 


0,0 


If  (j ,k)  4  (0,0)  we  have 


+  x{‘J+unl  +  l*+vnl  >9j+un,*+vn  =  0 

and  so 

gj+un,k+vn  X{(j.+U7ll2  +  U+vnl2  ) 

_  (-  *jk)  1 

9  j  ,k  |.9J  ,k  n  j  1  1  jk 


where 


u,v  { I j+un I 2 + I k +vn I 2 } 


Consequently 


*  jk  rj  ,k 
gjk  n  A+r.. 


t  j  iJt  *  0  1 1 »  .  .  .  i n~l 


j  ,k 


and 


'j+un ,k+vn 


*  i 

-J± 

n 


{  I  j+un  |2  +  I  Jt+vn  |2 


k+r 


j  'k 


The  expected  integrated  mean  square  error  of  <y  ( t ,  s 
as  an  estimate  of  f  is 


and 


E 


,*  =  “« 


1 9j.k-fi'k 


+  O 


Now 


2 

o 


(»0.0) 


2 

n 


f  9 


00 


=  / 


00 


Since  g  =0  for  (u,v)  ^  (0,0)  ,  o2  (g  )  =  0 

un ,vn  un  , vn 

(u,v)  i  (0,0)  .  If  (j  ,k)  *  (0,0)  ,  j,k *0,l,...,n-l 


£  (x  .  /n )  =  ?  . 

J  »*  J* 


For  this  range  of  (j,k)  we  have 
£  9 

and 


-1 


=  {  I  j  +  un  I  2  +  I  k  +  vn  r  )  (A+r  ) 
j+un,k+vn  J  jk 


a  (g  )  -  {  I  j+un  1 2  +  Ik+vn  |2}  2 (A+r  )  2  ~  . 

’ _7  +ur , k  +vn  J  jk  i 


THE  RESULTS.  The  expected  integrated  mean  square 

error  is  the  sum  of  the  integral  of  the  variance  of 

2 

/  o  I  g  (t , -t )  ]  dtdT  ,  and  the  integral  of  the  bias  of  g 
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•v-  .■»  . 


Hi  I 


squared.  Each  of  these  terms  will  be  estimated  under  as¬ 
sumptions  which  imply  that  they  tend  to  zero  as  . 

The  first  result  is  concerned  with  the  variance. 


THEOREM  1.  Let  f£C  and  satisfy  the  boundary  con¬ 
ditions  (A) .  If  g  minimizes  (3)  among  all  functions 

11  2 

satisfying  (5),  then  //  o  [  g  ( t  ,  t  )  1  d  t  dt  ■*  0  as  n-**  and 

°°  2 

A  (n)  -»  0  if  and  only  if  X  (n)n  ■*  ■»  .  If  this  is  so  then 


1 1 


( 8 )  ' j  o  | g  (t , t ) ] dtdt  = 


;; 


dxdy 


7  7  7  2 

n  X  (n)  -=>  ( 1  +  x  +  y  ) 


+0 ( n~ 2  X  (n) _1) 


The  integral  of  can  be  written  as 

n- 1 


— g  ^  E*  L  llj  +  unl2+lA+v/j|‘}2(A+r^)2 

n‘  n  jik- 0  u,v=-»  J 


where  E'  denotes  the  sum  deleting  (j,k)=( 0,0)  .  This 
can  in  turn  be  shown  to  be  equal  to 


2  2 

+  ~  E'  (A+r  )  2  E  {  I  j  +  ur  I2  +  I  k  +  vn  1 2  }  2  + 

n*  n  ljl,UtlSj  J  u,v=-“ 

,  -2  -  6- 
+  0  (  A  n  ) 


The  sum  can  be  written  as 


+  £?  ~  U  +  rjJc)"2 


{j  +k  ) 


2,2  +  ejk 


.  sL  r  »♦',*>  ‘  ~  — 
»2  m.ms£  12  **  1 


+ 


with  the  term  e  =0(n”  )  uniformly  in  j.A 
3  * 


r  *  ojrT*  z 


*-  ->.0tn-M>-2) 


Also 


1 


C  j 2  +A-2 )  2  (A  +  r  )2  (1+A  (j2+Jc2)  )2 

J  " 


with 


!d  I  < - - - - -  (j2+*2)  [{  (j+n)2-t-Jc2}  \{j 

J  a  +  >(/+*)) 

and  /c  a  constant.  Now 


"  ljl,l*ISy 


r' 


(  7  2  2  ) 


I  jf  I  ,  I  A  l<”  <1+x</+*2))3 


1  (n  M  X  1 )  . 


and  so 


+  (Jc+n)  ‘ 


Since 


of  -  1 _ 

71  ‘  Ij  I,  1*1*2.  (1  +>lj‘+*2)) 


% 


dxd  y 


7  2  2 

( 1  +  X  +  y  ) 


the  theorem  follows. 

Let  us  note  that  if  f£cJ  and  satisfies  the  bound 

ary  conditions  (4),  the  following  asymptotic  estimates 

hold  for  the  Fourier  coefficients  f  ,  : 

Jk 


(9  '  ) 


Jk 


(  2  r.  l  j  )  ‘ 


**  +  £' 


1  1  1 


as  I j  I  -  «  with 


a  =  /  e~2TlkJ{cf(l,i )-D  /(0,T)/d7  , 

*  0  £  c 


(9") 


Jk 


1 

(2  t i k)  2 


b  . 
J 


+ 


as  I  k  I  -*  »  with 


b  =  !  e  "  (t,l)-D,f  (t,C)  :dt  , 


Jk  [2r.ij)2  {2  r.ik) 


1 — r  :(D  c z  f  (0,t)) 

,  v  2  t.  t  .  t  T 


-  ID'  D  f(t,l)-D\D  f{t,0))\:  +  C 


(9'"! 


as  |jl,|Jtl  -*  ®  .  This  suggests  considering  the  following 
more  general  set  of  conditions: 


(10')  Ujk'2  =  Ul"2nlakl2  +  odjl'2") 

as  I  j  I  -*  «  , 

do”)  if  j2  =  ur2nii.  i2  +  o(i*r2n) 
j*  j 

as  I A  I  -+  *  , 


do"')  if.,  r  =%u*r2n  +  o(u*r2n) 

J  * 

as  I  j  I  ,  Uc  I  •*  «  ,  all  with  ^  >  n  >  • 

THEOREM  2.  If  >-*-0  and  n  \  ■*  »  as  n-*<*  ,  then 
der  the  assumption  that 

f  j  k  =  0  ( j  1  E)  as  I  j  I  -»  * 

(11)  f  =  CM*”1’  C)  as  1*1  -  «> 

J  * 

f  .  .  =  CUjk)  1  E)  as  I  j  I  ,  I  *  I  •+  » 

J  * 

for  some  t>0  ,  we  have  the  integrated  squared  bias 


(12) 


/  :  i  £5  (t ,i ) */  It  it )  |  d t d T  = 

0  V 


\j  I  ,  I*  l<^ 


J* 


x3  II2 +*2) 
(1+x  (j2+*2) )2 


un  - 
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Under  the  condition  (10)  the  integrated  squared  lias  i 


1 


(13) 

y 

y  [z  \.ki'  +  z  1**1"} 

u  -2  n 

J  — - n-j  dx 

(l+x^)? 

as  n-»<* 

.  I  n 

the  special  case  of  condition  (9)  we  have 

(14) 

c- 

U 

0  1  9 

l*kr  =  -r  '°tf  (1'T) "■Dt/ (0'T)  l  dt 

and 

(14  ’  ) 

\- 

L~ 

o  ^ 

:£■,  j‘  =  /  |c  /(t,l)-0  /(t,0)  j^dt  . 

2  T 

First  notice  that  the  integrated  squared  bias  can 
be  written  as 


{  .  ,-f 


n  -  x 

K~  ' 


]  ,  k  = ;  u  ,  V 


Lul 


( ()+un) (i+vn) ^ ) (A+r  ) 

jk 


Our  object  is  to  approximate  this  expression  by 


the  error  in  the  approximation  can  be  seen  to  be  of  the 
order  of 


n  - 1 


(16)  I 


0  0  1 


r 


i  f 


j.*=‘ 


I  j  +  un  l>  — 


n  {  (j  +  Lml^  +  lJc  +  vn)^}*1  (i  +  r  . .  ) 


J* 


or 


lA-un  (>“ 


ri  —  jl 


i 


•+  t  -  _ _ 

.  ,  vn  ''  j  ,k=i  .  .  ,.,n  !  (j  +  un)‘  +  (*  +  vn)  } ( X+r  ) 

i I >-  J  I  j  +  u-  n  I  £  r 


or 


or 


I*  l>j 


I k +  vn IS; 


The  first  term  of  (16)  is  C  (n  4)  .  The  third  term  of 
(16)  can  be  shown  to  be  0 (n  ‘  ).  The  second  term  of 

(16)  is 


n-1 


j  ,k~ 


Ljl 


Inder  the  assumption  (11)  this  can  be  seen  to  be 
C  l r.  *“>  ‘  4  ‘  )  .  The  last  term  of  (14)  can  be  estimated  by 


It  is  useful  in  estimating  (15)  to  remark  that 


{ j  +  k‘  )  (k  +  r  )  l  +  \  (j 
J  * 


1  +  0 


+  k 


?  2 


I  ' 


+  — - -I  t 

j'+  (k  +  n)  ‘  1  > 


This  implies  that  (15)  equals 


(17) 


+  C 


\j  I  ,  I k  l< 


Mj  I,  Ik  l<" 


7  7  7 

f  .2  >  (  7  +K  )  + 

I  t  ,  k  I  2  2  o 

<n  JK  (l  +  >.  (j  +*'))* 


,fJ  k 


,  2  i(2  j  2  i .2 

n  n  +  n  n 

(i+i)5  +  (^)5  (i)2  +  (L,i)"J, 

j-j  r*  rt  n  x 


n  n  n 


The  second  term  of  (17)  is  clearly  C  (r,  * £  )  .  This 

gives  us  the  estimate  (12)  .  If  the  asymptotic  behavior 

of  the  Fourier  coefficients  f  is  prescribed  by  con- 

J  * 

ditions  (10) , 

I  U.  2  +*!>?-- 

.  .  .  n  3k  1  *x\j2  +k2  )  / 

\  J  '  —  2 

for  fixed  k  is  asymptotically  the  same  as 


I,  IS? 


(l+x‘) 


This  ana  the  corresponding  result  fcr  Z  |f 


Ik \<~ 


jk 


»*V 
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with  j  fixed  lead  to  (13).  When  n=l  (14)  and  (14') 
are  obtained. 


COROLLARY  1.  If 

(18)  |f  ..  I?  )2  <  » 

j  ,  k  Jk 

the  integrated  squared  bias  is  asymptotically  to  the  first 
order 


This  remark  follows  immediatley  from  Theorem  2.  Notice 

3 

that  for  a  function  f  e  c  ,  the  condition  (18)  will  be 
satisfied  only  if  the  boundary  conditions 


(19)  D tf  (1,t)  =  Dtf ( 0 , t )  , 

d  /  ( t ,  1)  =  r>Tf(t,0) 

are  satisfied  in  addition  to  (4) . 

If  (18)  is  satisfied,  the  optimal  rate  of  decay  to 

-4/3 

zero  for  the  integrated  mean  square  error  is  n  and 

_  o  / 

this  occurs  when  l (n)  ^ n  .If  the  conditions  (9)  are 
satisfied  the  optimal  rate  is  n  e / 5  and  this  is  ob- 
tained  when  >,  (n)  ^  n  .  Notice  that  this  slower  rate 

is  due  to  a  boundary  effect  just  as  in  the  one  dimensional 


case,  the  fact  that  the  boundary  conditions  (19)  are  not 
satisf ied . 
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